Papers with reasoning complexity
Answering Open-Domain Questions of Varying Reasoning Steps from Text (2021.emnlp-main)
Copied to clipboard
| Challenge: | a new benchmark is developed to answer open-domain questions from text . the system uses a single multi-task transformer model to perform all the necessary subtasks . |
| Approach: | They develop a unified system to answer directly from open-domain questions . they use a single multi-task transformer model to perform all the necessary subtasks . |
| Outcome: | The proposed system can answer open-domain questions on any text collection without prior knowledge of reasoning complexity. |
ReasonRec: A Reasoning-Augmented Multimodal Agent for Unified Recommendation (2026.findings-acl)
Copied to clipboard
Yihua Zhang, Mingfu Liang, Jiyan Yang, Rong Jin, Wen-Yen Chen, Yiping Han, Huayu Li, Buyun Zhang, Liang Luo, Luke Simon, Sijia Liu, Tianlong Chen, Xi Liu
| Challenge: | Recent advances in multimodal recommenders lack explicit reasoning and self-awareness of uncertainty. |
| Approach: | They propose a reasoning-augmented multimodal agent structured around a three-stage explicit reasoning pipeline. |
| Outcome: | The proposed agent improves ranking metrics and performance on four standard recommendation tasks across five real-world datasets. |
AlgoPuzzleVQA: Diagnosing Multimodal Reasoning Challenges of Language Models with Algorithmic Multimodal Puzzles (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing datasets focused on visual question-answering focus on visual, language, and algorithmic knowledge . a new study examines the performance of multimodal language models in solving algorithmic puzzles . |
| Approach: | They propose a dataset to test the capabilities of multimodal language models in solving algorithmic puzzles. |
| Outcome: | The proposed dataset is generated automatically from human code. |
Token-Budget-Aware LLM Reasoning (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to enhance reasoning capabilities of large language models incur significant overhead in token usage, leading to increased costs. |
| Approach: | They propose a token-budget-aware LLM reasoning framework that adjusts the number of reasoning tokens based on the reasoning complexity of each problem. |
| Outcome: | The proposed method reduces token costs in CoT reasoning with only a slight performance reduction. |
GeoChain: Multimodal Chain-of-Thought for Geographic Reasoning (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Leveraging 1.46 million Mapillary street-level images, GeoChain pairs each image with a 21-step chain-of-thought (CoT) question sequence (over 30 million Q&A pairs). |
| Approach: | They propose a large-scale benchmark for evaluating step-by-step geographic reasoning in multimodal large language models (MLLMs) they pair each image with a 21-step chain-of-thought (CoT) question sequence (over 30 million Q&A pairs) |
| Outcome: | The proposed benchmark pairs 1.46 million images with a 21-step chain-of-thought (CoT) question sequence (over 30 million Q&A pairs) |
Small Models Struggle to Learn from Strong Reasoners (2025.findings-acl)
Copied to clipboard
Yuetai Li, Xiang Yue, Zhangchen Xu, Fengqing Jiang, Luyao Niu, Bill Yuchen Lin, Bhaskar Ramasubramanian, Radha Poovendran
| Challenge: | a small learning gap exists between large and small language models . long CoT data and large model responses are not beneficial for small models - a problem that may be due to the small student model's ability to handle distribution shifts. |
| Approach: | They propose a mix distillation strategy that balances reasoning complexity by combining long and short CoT examples or reasoning from both larger and smaller models. |
| Outcome: | The proposed strategy outperforms training on large and small models on short CoT and small model CoT. |
IterCOMP: Reasoning-aware Adaptive Prompt Compression for Multi-hop Question Answering (2026.acl-long)
Copied to clipboard
| Challenge: | Existing prompt compression methods are designed for single-turn queries and fail to capture interdependent reasoning steps. |
| Approach: | They propose a unified, training-free prompt compression framework that integrates multi-hop reasoning within an iterative compression loop. |
| Outcome: | Experiments on MusiQue, 2WikiMultiHopQA, and HotpotQA show that iterCOMP achieves significant improvements in Exact Match and F1 scores while reducing the token budget. |
Thinking Economically: A Hierarchical Framework for Adaptive-Complexity Reasoning in LLMs (2026.findings-acl)
Copied to clipboard
Yubo Gao, Haotian Wu, Hong Chen, Junquan Huang, Yibo Yan, Jungang Li, Zihao Dongfang, Sicheng Tao, PS Tan, Jie Zhang, Xuming Hu
| Challenge: | Existing efficiency methods for Chain-of-Thought (CoT) generate excessively long rationales without commensurate accuracy gains. |
| Approach: | They propose a training framework that operationalizes this principle through coarse-to-fine budgeting. |
| Outcome: | Experiments on GSM8K and MATH500 show that HAB surpasses standard CoT in accuracy and reduces token usage, achieving stronger performance-efficiency trade-off than baselines. |
MMTabReal: Real-World Benchmark for Multimodal Table Understanding (2026.findings-acl)
Copied to clipboard
| Challenge: | Multimodal tables are ubiquitous in real applications but are difficult to evaluate in multimodal large language models. |
| Approach: | They propose a multimodal table benchmark that compares 500 real-world tables with 4021 question–answer pairs. |
| Outcome: | MMtabReal spans four question types, five reasoning categories, and eight structural archetypes. |